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Abstract. We prove the smoothness of abnormal minimizers of subriemannian manifolds 
of step 3 with a nilpotent basis. We prove that rank 2 Carnot groups of step 4 admit no 
'^s^'. strictly abnormal minimizers. For any subriemannian manifolds of step< 6 we show all 

abnormal minimizers have no corner type singularities. 

o: 

(N 

■ 1. Introduction 

■ In this paper we are concerned with the smoothness of abnormal subriemannian geodesies, 
CN I which is one of the fundamental problems on subriemannian geometry. We will give sev- 
eral regularity results. By developing a reduction argument we will prove all abnormal 

^ ■ minimizers of subriemannian manifolds of step 3 with a nilpotent basis are smooth. By 

Q ■ proving that abnormal minimizers in any rank 2 Carnot group of step 4 are integral curves 

(•H ! of left-invariant vector fields, we clarify that the example by Gole-Karidi [21 J is in fact not 

\ a strictly abnormal geodesic. We also generalize the main result of Leonardi-Monti CTll to 

any subriemannian manifolds of step< 6. 

A subriemannian manifold is a smooth, connected n-dimensional manifold M with a 
rank k subbundle or distribution A c TM on which a smooth inner product g.,r is endowed. 
>• I i^,gsr) is called a subriemannian structure on M and A horizontal bundle. For q e M, 

■ define 
00 ■ 

^ : = span{[Xi, [X2, [• • • , • • • ]]](^) : Xjiq) 6 A„ j = 1, • • • , /}. 

■ We say A is bracket generating at qo e M if there exists an integer I such that A^^^^ = Tq^M. 
O I The least such integer I is called the degree of nonholonomy or the step of A at qo. In this 

paper we always assume A satisfies the bracket generating condition at each point of M. We 
say that q^ is a regular point if the integers ni{q) = dim A^ (/ = 1 , • • • ) remain constant for 
q'm some neighborhood of q^; otherwise we say ^0 is a singular point. We call (ni, • • • , n/) 
rN ! the growth vector of A at q^. It follows from the Chow-Rashevskii connectivity theorem 

\ that for any given points p,q e M there always exists at least a horizontal curve connecting 

p and q, see Il20l [331 . Here a horizontal curve is by definition an absolutely continuous 
curve 7 : [0, 1] — > M such that y{t) e A.y(t)M whenever y(t) exists. Thus one can define a 
natural distance: 



(N 



dccip, q) = inf ( ^[gJ^^j)dt 
Jo 

where the infimum is taken among all horizontal curves y such that y(0) = p and y{\) = q. 
dec is called the subriemannian or Carnot-Caratheodory distance of (M, A,g„-). A subrie- 
mannian geodesic (or minimizer) is a horizontal curve locally realizing A^c- It is not difficult 
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to prove that any two sufficiently close points can be joined by a minimizing geodesic. If 
(M, dec) is complete, there is a minimizing geodesic connecting any two given points. The 
topology of (M, dec) coincides with the original one of M. 

There are two types of subriemannian geodesies. Normal geodesies are smooth because 
their (Hamiltonian) lifts in the cotangent bundle T*M satisfy the subriemannian Hamilton- 
ian equation. The case of singular or abnormal geodesies is much more delicate. Singular 
geodesies may be normal and those which are not normal for any lifts are called strictly 
abnormal geodesies. Since Montgomery [32] (in 1991) discovered a smooth singular mini- 
mizer, there are many substantial results on regularity of singular subriemannian geodesies 
so far. For rank 2 distributions Liu-Sussmann [,28 | proved that regular abnormal extremals 
are locally length minimizing, see also ffOll . It was proved [11] that there are no strictly 
singular geodesies in subriemannian manifolds of medium-fat type (including step 2 case). 
For a class of regular subriemannian manifolds, Leonardi-Monti [27] showed that length- 
minimizing curves have no corner-type singularities which in particular implies that all 
singular geodesies in Carnot groups of rank 2 with step< 4 are smooth. In [|38]| we proved 
that there are no strictly abnormal minimizers in Carnot groups of step 3. The problem 
is still open for general cases. One may see [[B IB IS [HI [II [13 [IS IID llll |23l IM IM 
|32l[36l[38l[39| (the list is not complete anyway) and references therein for the state of the 
problem of subriemannian geodesies. We refer to [|21 [§1 [I5] [T71 [3T1 as book references on 
subriemannian geometry and nonholonomic control. 

Our heuristic idea is that if an abnormal geodesic is smooth then it must be an integral 
curve of a (smooth) horizontal vector field which we try to roughly locate by a rank or 
dimension reduction argument. To this end we need first order necessary conditions (by the 
Pontryagin Maximum Principle, PMP for short) and second order conditions (by Agrachev 
and his collaborators) for optimality of abnormal curves. The Hamiltonian formulation of 
these conditions is very helpful for our purposes. First order conditions by PMP implies that 
the Hamiltionian lift T of a shortest geodesic y of (M, A, g,,-) is also a length minimizer of 
a subriemannian structure {T*M, A'^, g*^) in the cotangent bundle, where A'^ is a distribution 
generated by a basis of vertical bundle (V){T*M) and Hamiltonian lifts of all horizontal 
vector fields of A, and ^* is the puUback of g^r by the canonical projection n : T*M M, 
see Definition 12.11 and 12.71 Here the Hamiltonian lift X'' is also called the complete lift 
of X 6 J^(M). Note that if A is bracket-generating so is A' . Second order conditions 
tell us that Y 6 (A^)^ := {co e A\M) : co(X) = 0, VX 6 Y(A^)} if y is a strictly abnormal 
geodesic. We will prove that the latter condition, also called Goh condition in the literature, 
implies that T is actually locally a subriemannian geodesic of a subriemannian structure 
(M' , Aj, ^* ), where M'' is a submanifold of T*, A^' is a rank (k - 1) (essentially) subbundle 
of A'^ possibly with step more than that of A. To guarantee the step of Aj is not more than 
that of A, we impose a nilpotent condition on A, that is, we assume A has a nilpotent local 
basis, see Definition 15.51 Under the nilpotent condition, in the step 3 case we can reduce 
the regularity of abnormal geodesies to rank 2 case, see Theorem 15. 61 

We will pay more attention to some subriemannian structures with step more than 3. 
Since Carnot groups are tangent cones at regular points of a subriemannian manifold, the 
smoothness of abnormal geodesies in Carnot groups may shed some light on the regularity 
of the shortest geodesies in general subriemannian manifolds. As done in ETl , by nilpo- 
tent approximations if the following "cornered-interval" (precisely its horizontal lift, see 
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Subsection 16.21 for detail) in a free Carnot group G of rank 2 with step r > 3 
(1.1) y\t)-- 



te2, te[0,^] 
1 1 1 



is not an abnormal (global) minimizer, then abnormal minimizers in regular subriemannian 
manifolds of step r have no corner type singularities. By the exponential mapping we 
identify G with its Lie algebra b = V^®- ■ -V (satisfying V = [V\ V'^] for i = 2, - ■ ■ ,r and 
[y\ V] = 0). Here = span{ei, ^2}- The shortening technique used in |I271 is limited to a 
class of regular subriemannian manifolds satisfying [A', A^] c A'+^"^ for /, j > 2,i + j > 5. 
In this paper we employ the system of abnormal equations in Camot groups to obtain the 
non-optimality of the horizontal lift of in (11.11 ). More precisely we will prove that the 
horizontal lift of is not an abnormal curve if and only if r < 6, see Theorem 16. 3[ From 
second order conditions for optimality of abnormal curves (see Theorem 14.51 ). we will 
prove that there are no strictly abnormal geodesies on rank 2 Camot groups with step 4, see 
Theorem l6.2[ 

To include those non-regular subriemannian structures we will need a lifting lemma 
or argument which was firstly introduced by Rothschild-Stein [34] in nilpotent analysis 
and lately was used to study subriemannian geometry and nonholonomic control by e.g. 
Bellaiche lfT4l and Jean [|25ll . If qo is a singular point of A, one can desingularize it by 
locally lifting (M, A, g.,r) to a regular subriemannian manfold which has the maximal growth 
vector (among all subriemannian manifolds with the same step and rank), see Lemma [X4l in 
Section [21 It is remarkable that the lifting or projection operator preserves the regularity of 
subriemannian geodesies. Several lifts we use here are actually more or less generalizations 
of the notion of Riemannian submersions. 

In next section after giving some basic concepts on subriemannian geometry from the 
Hamiltonian viewpoint, we will concentrate on nilpotent approximations and the lifting 
lemma. We will discuss in Section |4] some basic facts on second order necessary condi- 
tions for optimality of abnormal curves. In Theorem 14.51 we will derive the Goh condition 
and generalized Legendre condition in Camot groups. Section [5] is devoted to the rank re- 
duction argument and proving the smoothness of strictly abnormal geodesies in nilpotent 
subriemannian manifolds of step 3. In Section [6] after proving the normalness of abnor- 
mal minimizers in rank 2 Camot groups of step 4, we consider the non-optimality of the 
horizontal lift of in (|l.li) in free Carnot groups of rank 2. 

Acknowledgments. Part of the work was done when the first author visited Department 
of Mathematics, University of Notre Dame. We dedicate this paper to Professor Jianguo 
Cao who recently passed away. We thank him for his help and the first author thanks the 
staff of the Math Department for their hospitality. 



2. Some basic concepts from Hamiltonian viewpoint 

The cotangent bundle n : T*M M has a natural symplectic structure with the canon- 
ical symplectic form Q = da where a is the tautological (or Liouville) 1-form which is 
defined as follows: 



a(x. A) = n*A, V/l G T*M, Vjc e M. 
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A chart x = (x^, - ■ ■ ,x'') on M induces a coordinate chart • • ■ ,x",Ai, - ■ ■ , An) on T*M. 
In such coordinates we have the formulas a(x,A) = l^l^-^Aidx' and Q.(x,A) = l,"^^dAiAdx' 
where A = YJ^^Aidx' e T*M. 

— > 

Given any smooth function h 6 C°°{T*M) on T*M, its Hamiltonian vector field h is 
the vector field V such that Q(K T) = -Th for any T e ^(T*M). Given X e ^{M), let 
hx{x,A) =< A,X(x) >,A e T*M. The Hamiltonian lift or complete lift of X is defined as 

X' := hx-ln above coordinate charts, 



5jc' ^ '5jc' dAi 

where = ZLi ^'(^)^- ^ vector field V e 3^{T*M) is called projectable if there 
exists X 6 jr(M) such that X" = V. For any / 6 C"(M), X 6 ^(M) we have 

(2.2) = f{x)X\x, A)-y< A, X{x) > 

^ Ot oA; 

The vertical lift of a 1 -differential form to(x) = 0Ji(x)dx' 6 A'(M) is defined as 
a)^(x,A) = Z"=i ^((-^)^ 6 'f(T*M). It is easy to check the following properties (see 
e.g. f.26. Chapter 5]): 

(/a;r = /W, [co\r]=0, 

(2.3) = (Lza;)\ 

7r,(X') = X, [r, r] = [X, Yf, 

where co,tf/ e A^M); / 6 C~(M) and /^(O) := /(;r(0)); X,Y e ^(M) and Lx is the Lie 
derivative along X. 

Definition 2.1. Let = {Xi , • ■ • , X„} be a local frame of TM. Denote by ^ = {aj\- ■ ■ ,oj"} 
be its dual coframe of T*M. Define T, = {co'Y, then ^ = {Ti, • ■ • , r„} is a frame of 
the vertical bundle y(T*M) and {X^, ■ • ■ Ti, ■ • ■ , r„} is a basis for T{T*M). One can 
introduce another coordinate system on T*M adapted to the given moving frames. Let 

= Z"=i^/^- For O = (x,A) e T*M with A = Z"j=i^jdx^ e T*M. The new coordi- 
nate system for A is the unique n-tuple ^ = (^1, ■ • ■ ,^„) such that A = J^'j^^ ^jOJ^x). Thus 

= 2"=! XiAj and ^ = 5]^, ^ = 1, ■ • ■ , n,5 = (5p is the inverse of the matrix (XJ), 
that is, 2"=i = 2"=i B'.Xi = 5/ for /, 5 = 1, • • • , n. We have 

(2.4) A = r^,^= 1,... 

In fact, from oj' = 2"=i and ^ = X}|-, it follows T, = (ci>0" = 2"=i ^^^^ = 

2ji=lK2jj=l^j^i)g^^ - g^^- 



Lemma 2.2. Le? 3' he as in Definition \2.1\ For any Si, s, 

n 

\X:^,TA = Yu^oj\[X-M)T,. 



~s=\ 
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Proof. By direct computation, we have 

L si \ x,^ J ZjI ^1 ^ ' dxJ dAi 

j,t=i ^ ' J 



n n 

J=l j,t=l 

n ( n 



V i,r=l 



d 



dX' 



S=l \j,t=\ i,t=l ) 



s=\ i=\ 



£<a>^[Z„Z,J> — 



□ 



There exists a natural Poisson structure on T*M: for any f,g & C°°(T*M), {f,g} := 
— > 

Q(h f, hg). The closedness of Q. guarantees that (C°°(T*M), {■, •}) is a Lie algebra. It is 
easily to see Z'Tiy = {hx,hY} = h[x,Y],'iX, Y e ^(M). The following immediate facts will 
be frequently used. 

Lemma 2.3. Let A = {Xi,-- • ,Xk} and A' = A'~^ + [A\A'~^] (A^ = A). Assume A' is a 
distribution with a local basis {Fi, • • • , F„.}, / = 1, • • • , r. 

(1) (A'y = {<D e T*M : /jy.(O) = 0,7 = 1,- • • is asmoothsubmanifoldofT*M. 

(2) If r : [0,1] ^ (A'y(i > 2) is a Lipschtiz curve with Y(0 = u'(t)X'^(r(t)), a.e. 
t e [0, 1] for some u e L~[0, 1]. Then 



= mhy^im) = J] MXO/jK,y,](T(0) 



(2.5) 



J] u\t) (A(t), [X,(j(t)), Yjiym) = (m, [7(0, Yjijm) 



s=l 



a.e. t e [0, 1] and j = 1, • • • where T = (y, A). 

(3) LetO€ A'-\ X^(^) G T^A'-\ / = 1, • • • , i/anJ onfy j/O G A'. 

For technical convenience we fix xo g M, a small neighborhood ^ of xq and choose an 
orthonormal frame {Xi, • • • of A in i^. We point out that properties of subriemannian 
geodesies are independent of the choice of orthonormal frames. An absolutely continuous 
curve 7 from xq in ^ is horizontal if and only if it satisfies the Cauchy problem 

k 

7(0 = «'(0^;(r(0) a. e. ? G [0, 1] 

7=1 

lr(0) = xo 

for a function u = (a\ •••,«*) G L^([0, 1], R*"). Here u is called the control of 7. We 
may assume u G L^([0, 1],R*) (L^[0, 1] for short) without restriction. Let Ur = {u £ 



(2.6) 
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L^[0, 1] : ||m|| = r} be the sphere of radius r in L^[0, 1], where ||m|| is the standard Hilbert 
norm of LjlO, 1]. When r is small enough, solutions of equation (12.61) are defined for all 
t e [0, 1]. The end-point mapping S' : w ^ y(l) is a well defined smooth mapping from 
a neighborhood of the origin of Ll[0, 1] into ^. The length of y satisfies the Cauchy- 

Schartz inequality ^(r) = (z5=i("^'(0)^) dt < \\u\\ where the equality holds if and only 

if u is normalized, by which we mean 2y=i("^(0)^ = c for a constant c. Note that the 
length is independent of the parametrization of the curve. Then dcc{xo,x) = min{||M|| : u e 
L^[0, l],S'{u) = x} and the minimum is attained at a normalized control. A normalized 
control u is called minimal for the system (|2.6I) if dcdxo, (S'{u)) = \\u\\. So a minimal control 
corresponds to a shortest geodesic and vice versa. The Pontryagin Maximum Principle (or 
the Lagrange Multiplier Principle) gives first order necessary conditions for a control to be 
minimal: if m e ^^[0^ 1] is minimal then there exist Lagrange multipliers 6 T^^^^^M\{0}, 
such that 

(2.7) A^Ducff = vu 

where v = or 1, is the Frechet diff'erential at u of cS' and u in the right hand side is 
identified as an element in the dual space (L^[0, 1])*. A control u satisfying (|2.7I) for some 
nonzero is called an extremal control and the associated curve is called an extremal tra- 
jectory. We say an extremal control (or the associated trajectory) is regular if w is a regular 
point of S"; otherwise they are singular or abnormal. If v = 1 in (12.71 ). then the extremal 
control u is called normal. Regular extremal controls must be normal. Abnormal extremal 
controls which are not normal for any nonzero multipliers are called strictly abnormal. 

For t,T e [0, 1], let P[ : y(T) yit) be the Lipschitz flow of nonautonomous vector 
field 2?=i u'{t)Xi in (12.61 ). Assume u = (m\ • ■ • , a*") is a minimal control satisfying (12.71 ) 
for some /J^ 6 T*M\{0]. Let A{t) = (P]yA\ t e [0, 1). From the formula (see [i8i|2J) for 
v = iv\--- y)eLl[0,l] 

k ^\ 

(2.8) DuS{v) = y V(t)(PlUXt(ymdt, 

one may deduce that u'{t) =< A{t),Xi{x(t)) >, if v = 1; or < A{t),Xi(x(t)) >= if y = 0, z = 
1, • ■ • ,k. The last fact can be formulated as follows (see e.g. [2] for a proof). 

Theorem 2.4 (PMP). Let u = (u^, - ■ ■ ,u'') e L^IO,!] be a minimal control ( the associated 
trajectory is y). Then there exists a Lipschitz curve A{t) e T^^^^M\{0} such that T{t) 
(}'(/). A{t)) satisfies 

k 

(2.9) t(0 = Yj u\t)XiO'{t)\ a.e. t e [0, 1] 

1=1 

and exactly one of the following conditions is satisfied: 

(1) < A(t),Xi(y(t)) >= u'(t), / = 1, • ■ • Vf 6 [0, 1]; 

(2) < A(t), Xiiyit)) >= 0, / = 1, ■ • ■ , fc, 6 [0, 1]. 

Definition 2.5. We caU the curve Y subject to (|Z9l ) and T(l) = (7(1), A^) is the Hamiltonian 
lift of 7 satisfying (|2.6I) . Of course, there may exist infinitely many such lifts depending on 
A^ G (Im(D„(l'))"^, but we abuse the name and call any of them as "the" Hamiltonian lift. 
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Remark 2.6. The condition (2) in Theorem 12.41 just means that the Hamiltonian lift of 
an abnormal geodesic is a Lipschtiz curve in A-^. It is independent of the subriemannian 
metric g^, - If w is a normal minimal control, then u satisfies the condition (1) in Theorem 
I2.4[ Putting u into (12.91 ) we see that the right hand side of (12.91 ) is just 

k k k 

^ < A(t), Xiiyit)) > x^int)) = hxmAm) = J] ^('^■(O) 

(=1 (=1 1=1 

where H = ^ X?=i ^| subriemannian Hamiltonian. So the Hamiltonian lifts of normal 
geodesies are integral curves of a smooth Hamiltonian vector field. 

Definition 2.7. Motivated by Theorem 12.41 we lift the subriemannian structure (M, A, g^r) 
to the cotangent bundle T*M. Assume A = {Xi, ■ ■ ■ ,Xk} be a regular distribution. Let £/ 
be a frame Lie-generated by {Xi, ■ ■ ■ ,Xk} and £^ = {Ti, ■ • ■ , T,,} be as in Definition 12.11 
We construct A' := spanjXj, ■ • ■ Ti, - ■ ■ ,T„}. Define an inner product ^* on A' by 
declaring Xj, • • • , X^, Ti, - ■ ■ , r„ orthonormal. From Lemma[2]8](r*M, A'^, ^* ) is a regular 
subriemannian structure. 

Lemma 2.8. (1) A'^ is independent of the choice of local frames of A and it is a {k + n) 
dimensional distribution. 

(2) dim[(A'y\(A%^) = dim ((A)'\(A)'-i) /or any Oq e n-\xo), i > 2. So A' is also a 
regular distribution. 

Proof. (1) Let {Yi, - ■ ■ , 7^-} be another local frame of A in ^ c M. Then there exists 
a nonsingular smooth k x k matrix (aj) such that F, = 2j=i ^j^j' ' = 1, • ■ • From 
(1X21) we have Yf = ^ii + Zti b^Tj, for some b^ e C^in'^^)), i = I,-- - ,k. 

I J I. I J J 1 I J I 

So span{y[, • • • , y[, Ti, • • • ,T„} c span{Xp • • • , X^, Ti, • • • ,!„}. In the same way we also 
have span{Xj, • • • , X^, Ti, • • • , r„} c spanlYp • • • , 7^, Ti, • • • , So A'' is well-defined. If 
Zti A-' + 2"=i S^Tj = for gj 6 ^"(/r-H^)), then from n^Tj = 0, n^X^ = X,- we have 
2f=i /'(^, /l)X,(^) = for any A e T„M and x 6 ^ c M. Thus /' = in n'^^) which also 
implies that gj = in n-^ff). So • • • , X^, Ti, • • • , r„} is a local frame of A^ 

(2) Recalling for i > 2, (A')' = (A')'-^ + [A'' , (A'y-^]((A'y = A'), from (Q we can 
give a 1-1 correspondence between frames of (A')'\(A'')Jj^J and (A)'\(A)Jj^^ for any Oq 6 
7T~\xo). For F = (rj, • • • , r/)(rj 6 {1, 2, • • • , fc}, 5 = 1, • • • , m,- := (n,- - where = 
dim((A^)^J), let XI, = [X;„ • • • , [X^. , • • • • • • ]. Then {XS. , • • • , X^„, } is a basis of 

(A'')'\(A'')'"^ at Oo if and only if {X/i, ■ • ■ ,Xjn,,} is a basis of A'\A'"' at xq, where X/. = 
[Xrj, ■ • ■ ,[Xr' , Xr^] • • • ], 5 = 1, ■ • ■ , m,. □ 

I /-I ; 

Proposition 2.9. T/'y /5 a length-minimizer ( with the control u satisfying (12.61 ) ) in (M, A, g^r), 
then its Hamiltonian lift T is a shortest subriemannian geodesic in (T*M, A'^, ). 

Proof. Assume there exists a horizontal curve O : [0, 1] T*M in {T*M,A^,g*^) such 
that the length of O is less than that of Y and 0(0) = T(0), 0(1) = Y(l). Let 



k n 
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for V = (v^•■• ,v^) 6 L~[0, l],c = (c\--- ,c") e L~[0, 1]. Denote by 6 = 7r(0) the 
projection of O. Then 6 satisfies 6(0) = 7(0), 5(1) = y(l) and 

k 

m = J]v\t)x^(5(t)). 

i=l 

So £{6) < ^(O) < £{T) = i{y) which contradicts with the fact that y is length-minimizing. 

□ 

Remark 2.10. (1) Given a local frame {Y^ ■■■,¥„} of TM, let n = span{y[, • • • , 7^}. Then 
from (12.11) and (|2.2I) we infer that 11 is not well defined in the sense 11 depends on the choice 
of the local frame {^i, • • • , Y„}. One also may define IT' = spanjX'^ : X e J^{M)}. Again 
from (12.11) and (12.21) we can see 11' is a singular distribution. In fact, dim(n^^ q^) = n but 
dim(n;^^)) = 2nif i ^ 0. 

(2) For any point O 6 A-^, one can define the lift "/j a(3>) c T<^T*M of A: "/^a(^) = 
span{Xj, • • • ,X^}(0). The definition of /ja(3') does not depend on the choice of local 
frames of A because by dlj) (/'Xj + • ■ • + fXuy{Q>) = fX\(^) + ■ • • + /X^(0) for 
any O e A-^, /' e C"(M), i = I,-- - ,k. However X^'(O) is not in T^A-^ unless O is in (A^)-^ 
by (3) in Lemma [231 We will come to this point in Section |4l 

(3) There is another equivalent characterization of abnormal extremals. Because T^^iA-^) = 
— > 

(/z a)^(^) (the symplectic orthogonalization), Y e A"^ if and only if Y is a characteristic of 
n^^, that is, Q{V, Y) = for any V e TyA^. 

3. NiLPOTENT APPROXIMATION AND A LIFTING LEMMA 

3.1. Nilpotent approximations. We adopt the definition of privileged coordinates intro- 
duced by Bellaiche [14], see also [fTOl l22l . For the chosen system (Xi, ■ ■ ■ ^X^) in ^, let 
/o be the step of A at xq and (ni, • • • , n/J be the growth vector of A at xo 6 Define the 
weight sequence (^i < • ■ ■ < cu^ by setting ojj = s\i ris^i < j < n.,. For a smooth function 
/ on ff, we say that Xif,--- ,Xkf is the nonholonomic partial derivatives of order 1 of /; 
Xi^Xj^f, Xi^Xj^Xi^f, • • • is called the nonholonomic derivatives of order 2, 3, • • • , of /. If the 
nonholonomic derivatives of order < 5 - 1 of / vanish at q, we say that / is order > 5 at ^. 
A function / is of order 5 at ^ if it is of order > s but not of order > * -I- 1 . We say that local 
coordinates (xi, ■ • ■ , jc„) centered at xq are privileged coordinates at xq if the order of Xj at 
xq is equal to a*, for z = 1, ■ • ■ , n. 

There are several ways to construct privileged coordinates. Choose a sequence of vec- 
tor fields Fi, ■ • ■ ,Yn whose values at xq form a basis of T^^M such that 7, = Xj for i = 
I, ■ ■ ■ ,ni(= k), Ynj+i, • • ■ , Ynj_^j e A^'^'VA^ for j = 1, ■ • ■ , /q - 1- Then the canonical coordi- 
nates of the first type: (xi , • • • , Xn) xq exp (2"=i ^jYj) gives a privileged coordinate chart 
of ^ (shrinking if necessary) which maps Xq to 0, see e.g. O for a proof. 

Fix a system of privileged coordinates x = (xi, ■ ■ ■ , x,,) of with x{xo) = 0. For e > 
define dilations 6t;X = (e^'xi, • ■ • , e'^"Xn). We say that a polynomial is homogenous of 
weighted degree s if it is a linear combination of monomials x"' ■ ■ ■ x^", with ojiai -i- • • ■ -i- 
co„a„ = s. Its order at is equal to its weighted degree. A smooth function / is of order 
s if and only if f{5^x) = 6^/(jc) for any x,e > 0. For i = I, - ■ ■ ,k, Xj can be rewritten 
near as Xi(x) = J^^^iifijixi, ■ • ■ , Xj-i) + gij)-^, where is an homogeneous polynomial 

of weighted degree coj - 1 at 0. We define X, as 2"=i fij(xi, ■ ■ ■ ,Xj-i)-^. We call X,- 
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as the nilpotent approximation of X, for / = I, - ■ ■ ,k. Note (6^)i,Xi = eXi in iff. We 
use Xiid^x) = 6'\6^)^^,,Xiix) to extend Xi from ^ to R". Thus {Xi, • • • ,Xk} is a bracket 
generating system in R". We call G = (R", A,g^„.) the tangent space at xq of (M,A,gsr), 
where A = spanjXi, • • • ,Xk} and'^^^ is the metric such that {Xi, ■ ■ ■ ,Xk} is an orthonormal 
frame. 

The tangent space G constructed above depends on the choice of privileged coordinates. 
For intrinsic constructions we refer to ||29l 171121. When xq is a regular point, G = (R", A,'gsr) 
is a Carnot group where {Xi, ■ • ■ , Xi,} generating all left-invariant vector fieds, see e.g. [14J, 
lfT6l P. 191]. We recall a Camot group G is a connected, simply connected Lie group with 
a graded Lie algebra (called Carnot algebra) 

(3.1) b = e • • ■ e V', with V' = [V\ V'-^], [V\ V']=0,i = 2,-- - , I. 

The integer / is called the step of G. Let n, = Tj'j=i dim W for z = 1, • ■ • , /. We call 
(ni, • • • , ni) the growth vector of G or b. The following lemma can be easily deduced from 
the graded structure (13.11 ). see [UHl- 

Lemma 3.1. IfG = (R", A,'gsr) is a Carnot group, for the control u = (m^ • • • , m'') 6 L^[0, 1] 
in the following nilpotent system 

( 

yit) = y uKt)Xj{y{t)) a.e.te [0, 1] 

(3.2) ft 

.7(0) = 1 

we have u^t) = x^t), j = I,-- - ,k, a.e. t G [0,1]. Herey = (/,••• ,y^){Y e W"-, 
nii = ni — ni-\,m\ = ni = k) and y^ = (jc\ • • • , x'^). 

For an abstract Carnot group G, we sometimes by the exponential mapping exp : b ^ G 
identify G with its Lie algebra b ~ R" = R'"' © • • • © R'"' on which a group operation is 
endowed by the Baker-Campbell-Hausdorff formula. By Lemma 13.11 a horizontal curve 
y = (y 1 ^ . . . ^ y') in G is uuiqucly determined by its projection on the first layer y ' . So given 
y^ 6 L^IO, 1] with y(0) = 0, we call the y satisfying (13.21 ) the horizontal lift of y^ 

Nilpotent approximation (A,'^vr) can be regarded as the limit of the approximating sys- 
tem in a privileged coordinate chart at .x:o 6 ^: (A"^, g^^Xe 0), where A*^ = span{Xp • ■ • , X^}, 
X^j := 6(6^-i)i,Xj(j = 1, • • • ,k) and g% makes {X^, ■ • • ,X^} orthonormal. In fact, the con- 
vergence of lim^^o^j = Xj is locally uniform, see e.g. [I6l|271|2l. For a horizontal curve in 
(0", A, gsr), 7 : [-^, 5] ^ ^ with y(0) = xq, define its blow-up at xq as follows: 

y'{t) = 6,-iy{€t)),te[-^,^]. 

2e 2e 

Let u = (u^, - ■ ■ , M*") 6 |] be the control of y. Then by a direct computation we have 

k 

r{t) = J]u'(et)X%f(t)). 

!=1 

That is, y*^ is a horizontal curve of (A"^, g^^). 
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Theorem 3.2 (1271). Assume the normalized control u (i.e., y is parametrized proportion- 
ally to arclength) has left and right Lebesgue limits at 0, that is, 

u{T)dT = u+, lim 77"' I u{T)dT = 

1^^' Jo 

for'u+,'u_ e M/^. Then 



(3.4) r(t) = 



lim y^(0 = u+t, t 6 (0, 00) 

e— ►()+ 

lim y^t) = u^t, ? 6 (00, 0] 



exists and it is a horizontal curve in G = (R", A). Moreover, ify:[-j,^]^i^is length- 
minimizing in {A,gsr), y : [-^, 5] R." is length-minimizing in (A,'gsr) and thus'u+ --U-. 

We call a horizontal curve y : [-|, 5] & has a corner at Xq if the limits in (I3.3I ) exists 
and'M+, -TL are linearly independent. 



3.2. A lifting lemma. To include singular subriemannian structures, we need a lifting ar- 
gument to desingularize singular points. First we recall that a Carnot group G (or a Camot 
algebra b) is called a free Camot group (or a free Carnot algebra) with bi-dimension (^, /) 
if G (or b) has the maximal vector growth among all Camot groups (or all Carnot algebras) 
of step / with k generators. We denote by 7j(fc, Z) the dimension of a free Camot group with 
bi-dimension (^, /). 

The fc-dimensional distribution A of a subriemannian manifold (M, A,^^) is said free 
up to step Z at ^ e M if A has the maximal growth vector and step / at q. This implies 
the dimension of M is 7r(fc, Z) and A is free up to Z at each point of M (we always assume 
the bracket-generating condition of subriemannian manifolds). The following lemma is 
obvious. 

Lemma 3.3. If A is free up to step I at q e M (so free at each point), then the tangent space 
G at q of{M, A, g^^r) is a free Carnot group with bi-dimension (k, I). 

The following lifting method firstly introduced by Rothschild-Stein Il34l is very useful 
in nilpotent analysis and nonholonomic control. 

Lemma 3.4. Let Zo be the step of A at xo 6 M and {Xi, ■ ■ ■ ,Xk} be an orthonormal frame 
of A near xq. Denote byH = Hik, Zq) be the dimension of the free Carnot algebra of step Zq 
with k generators and by M be the manifold M x R""". Then there exist a neighborhood 
^ c M of(xo, 0); a neighborhood ^ c M ofxo, ^ x {0} c U; coordinates (y, z) on G; and 
vector fields on G 

~ " /) 

(3.5) Xi(y,z) = Xi{y) + ^ 4^^^)—. j=l,--- ,k 

j=n+l 

such that: 

(1) the system A = spanjXi, ■ • ■ , Xk} is free up to step Zo at each point in G; 
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(3.6) 



(2) ifue 1] is a minimal control of the system d2.(5l) in G then it is also a minimal 
control of the following system in G 

k 

q(t) = ^ u\t)Xj(q(t)) a. e. t e [0, 1] 

!=1 

[q(0) = (xo,0) 

(3) ifu is an abnormal (a normal) control of the system (12.61 ). then it is also an abnor- 
mal (a normal) control of the system (I3.6I) ," 

(4) if M is a Carnot group of step Iq and {Xi, ■ ■ ■ ,Xk} is a system of left-invariant 
vector fields generating the Lie algebra of M, then M can be a free Carnot group 
with bi-dimension (k, Iq) and the vector fields in (|3.5I) are left-invariant vector fields 
generating the Lie algebra ofM. 

Proof We refer to |[Il|25l for a proof of (1) and EH Chapter 17] for a proof of ©. The 
proof of (2) is similar to that of Proposition 12.91 since 7r*(X,) = Xi,i = I, - ■ ■ ,k where 
TT : G 3 (y,z) ^ y & G is, the natural projection. (3) from £'{u) = n o S{u) where S, S are 
end-point mappings for (12.61) . (13.61) respectively. In fact, we have D^S = n.^DuS' and thus 
if there exists A 6 T*^^^M\{0} such that ADuS' = (or = u) then A*Du(S' = (or = u), where 

A* = 7T*A e Tt M\{0}. □ 

#(u) 

By Lemma 1341 the regularity question of subriemannian geodesies of (M, A, g.^) near xq 
can be reduced to that of (M, A) near (xq, 0) where the tangent space is a free Carnot group 
by Lemma [231 

Example 3.5 ([|3l|3l). Let e{x) = dx^ - \{x^Ydx^ be the Martinet form in M?. The Martinet 
distribution is A = ker0 = span{Xi,X2} where 

d d {x^f d 



dx'^ ' dx^ 2 dx^ 

Every point in the plane [x^ = 0} is singular. We study abnormal geodesies starting from 0. 
The step of A at singular points is 3 and the growth vector at is (2, 2, 3), since [Xi, X2] = 
^^^^ [[^i,^2],^i] = It is easy to see x = {x^,x^, x^) is a privileged coordinate at 0. 
The nilpotent approximations of Xi and X2 are themselves: X\ = Xi, X2 = X2. Note that 
Xi,X2 is nilpotent of step 3 and 1-homogenous with respect to the dilation 6e{x^ ,x^, x^) = 
(6x\ 6X^, e^x^), we may add two variables {z\z^), choosing suitable functions to get 

,_+.^.(,,,)_ 

which generate all left invariant vector fields of the free Carnot group with bi-dimension 

(2, 3), see e.g. [fT6l P. 669]. Then X,(z = 1,2) are the desired vector fields in Lemma [341 
because X^ = Xi,X2 = X2. 

Let u{t) = (u^{t), u^it)) be an abnormal minimal control of the system 

2 

x(t) = ^ u\t)Xj(xit)) a. e. ? 6 [0, 1] 



Xi(x,z) = Xi(x) -I- aj(x,z)— + a^(x,z)—, i= 1,2 



(3.7) 



i=\ 

x(0) = 
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By Lemma I3.4[ u is also an abnormal minimal control of the free Carnot group (denoted 
by G(2, 3)) with bi-dimension (2,3). It is well-known (see e.g. Theorem 5.2] for a 
proof) that abnormal minimal controls in G(2, 3) has the form u(t) = (c\ c^) for constants 
c^ with (c^)^ + (c^)^ and all of them are normal. Among all these controls there is a 
special one u{t) = (0, c) for a constant c whose associated curve of the system (|3.7I) is 
the example discovered by Montgomery [32] and it is a strictly abnormal minimizer. 

Remark 3.6. The statement (3) of Lemma [J4l implies that strictly abnormal controls of the 
system (13.61) are also strictly abnormal controls of the system (|2.6I) . But from Example [331 
the converse is not true in general. 

4. Second order conditions for strictly abnormal minimizers and their versions on 

Carnot groups 

4.1. Goh condition on general subriemannian manifolds. Agrachev and his collabo- 
rators [SI [8l m [I2l developed second order necessary conditions for minimality of con- 
trols: for any minimal control u of the system (13.61) there exist Lagrange multipiliers e 
r*(^^^\{0} such that A^D^^ = vu(v=\or 0) and ind(^; u,A\v)<0 where ind((f ; u,A\v): = 
ind+(/i'Hess„^) - dim cokerD„(l', /l'Hess„^ is the intrinsic quadratic form defined by 

^^Hess„Aw) = A^dI^(w, w) - v\\wf, w 6 kerD^^T 

and 

ind+(i^Hess„fr) := sup (dim V : V c ker A,^,/l'Hess„fr(w) > 0, Vw e V\{0}] 

is the Morse index of the quadratic form A^HesSuS'. Note that ind((f ; m, /i^ v) < im- 
plies the finiteness of ind+(/l'Hess„^). Since normal geodesies are always smooth (see 
Remark [Z6l) . in the following we only consider strictly abnormal geodesies. The finiteness 
of ind+(/l'Hess„(l') for strictly abnormal minimal controls u implies the following important 
theorem. 

Theorem 4.1 ([SIH^)' Let u = (u^, - ■ ■ ,u^) be a strictly abnormal minimal control of the 
system (12.61) (its associated minimizer is y). Then there exists A^ e T^^^^M\{0} such that 
T(0 = {y{t),A(t)) 6 A;^(^^, where A(t) = (Pl)*A\t 6 [0, 1]), satisfies dU) and 

(1) (Ait), [Xi,Xj]{y{t))) = 0, e [0, 1], ij =!,■■■, k; 

(2) {A{t),[[y(t),X,(ym,Xriym) > 0, for a.e. t e [0, 1] and any v ^ X,{y{t)) := 
Y!i=iV%{y{t))forv = {v\--- 

Definition 4.2. The condition ([T]) in Theorem 14. H is called Goh condition. It is equivalent 
to (a priori T(f) 6 A;^^^^ by Theorem [231) 

(4.1) T(Oe(A%), V?6[0,l]. 

Any curve y of the system (12.61) whose Hamiltonian lift satisfies (14.11) is called a Goh curve. 
The condition (O in Theorem 14. H is called Generalized Legendre Condition. 

Theorem 4.3. Let y be a strictly abnormal minimizer of{M,A,gsr) (satisfying (12.61) with 
its control u), then its Hamiltonian lift T = (y,A) is also an abnormal minimizer in 
(T*M, A'^, g*^) and satisfies the Goh condition and generalized Legendre condition. More 
precisely, let A*{t) = 7T*A(t) e T*(T*M) and A = span{Xi, • • • then 
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(1) {A*(t),[X^,X'^j]{m)) = {A*(t),[X^\TMr(t))) = avre [0,1], ij= h---,k,s = 
1, • ■ • ,n; 

(2) (^*(0,[[t(0,x,^(T(0)],x^(T(0)]) > 0, (i*(0,[[t(o,r,,(T(0)],r,,(T(0)]) = o, 

{A*it),[[m,X^(rm,T,,irm) = Ofora.e. t e [0,1],/ = I, ■ ■ ■ ,k and su S2 = 
1, • • • , n, Vv 6 Here X'M^)) ■= v'X^(y(t))for v = (v\--- , v'). 

Proof. Because y is a strictly abnormal minimizer in (M, A,^,^), its Hamiltonian lift Y = 
(y. A) satisfies the Goh condition and generalized Legendre condition as in Theorem 14 .ll for 
some A^ 0. By Proposition l2.9[ Y is a length minimizer of {T*M, A'^, ) and satisfies 

k 

(4.2) m = u'mfiTit)), a.e. te[0,l]. 

;=1 

Let A*^ = n*A^ e T*^^^{T*M) and A*it) = (^/)* A*^ 6 T*^^^iT*M), where i^;)* : T{t) 
Y(t) is the Lipschitz flow determined by (14.21) . Then 

n*A(t) = n\Pl)*A^ = {P\ o n)* A^ = {n o ^l)*A^ = (^/)* A*^ = A*(t) 

and 

{^*\ A,^') = {A\n,D,,c^') = {A\D,(no^^)) = {a\D,,c^) = 0, 

where (f , are the end-point mappings for the systems (|2.6I) . (14.21) respectively. There- 
fore Y is abnormal in (T*M, A'^,^*). A direct computation shows A* verifies the desired 
conditions, since n-^X'^ = X, n-^T^ = for any X 6 J^{M) and s = I, - ■ ■ ,n. □ 

Remark 4.4. (1) Both the Goh condition and Generalized Legendre condition are indepen- 
dent of the subriemannian metric g^r- 

(2) Abnormal controls of rank 2 distributions automatically satisfy the Goh condition. 
In fact, if M = (u\u^) is an abnormal control with y its associated trajectory of the system 
(|2.6I) where k = 2, then by Theorem 12.41 the Hamiltonian lift Y = (y, A) of y is a Lipschtiz 
curve in A-^. So from Lemma 123) we have 

-u\t) < A{t), [Xi,X2](y(0) >= u\t) < A(t), [Xi,Z2](y(0) >= 

which implies < A{t), [Xi,X2]{y(t)) >= a.e. t e [0, 1] (we do not consider trivial curves). 
By the continuity of A we get < A(t), [Xi,X2]{y{t)) >= 0, V? e [0, 1]. 

(3) Assume A^ is a smooth distribution. Since the Hamiltonian lift Y = (y, A) of a Goh 
curve y is a Lipschitz curve in (A^)"^, from Lemma [23] 

(4.3) {A(t), [y(0, [Xi,Xj](r{t))]) = a.e. t e [0, 1]. 

4.2. Goh condition on Carnot groups. In this subsection we shall derive second order 
conditions for minimality of abnormal controls in Camot groups. There are two ways to do 
it. As in [[38| one may use the formulation (see Lemma [3711) of the end-point mapping in 
Camot groups to directly derive these conditions. Here we will just translate the conditions 
in Theorem 14. II to the setting of Carnot groups. 

As pointed out in Section [3] we identify (via the exponential mapping exp : b ^ G) a 
Camot group G with its Lie algebra b, or the tangent space TiG of G at the identity 1. Any 
V e r^G is identified with some ^ e b such that {Lq)^ i^ where : G 3 p ^ q.p is the left 
translation by q. The diff'erential of the exponential mapping denoted by 

(4.4) df exp : b 9 ^ ^ (^exp(-^))*,exp^ exp^^^(Z:) 6 b 
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is an isomorphism on b, since G is nilpotent. In the following we will use the fact (exp ^)"^ = 
exp(-^) to denote (exp^)"' by - exp^. We have the following formula 

(4.5) d,exp = = Id - y ^^(ada-\ 

ad4^ ^ s\ 

s=2 

where r is the step of G. Its inverse is 

(=1 

where Bi = -\,B2 = ^,Bi, = 0,^4 = • ■ • are Bernoulli numbers. We choose a basis 

{ei, • • • ,ek,--- , en) of b such that X;(<?) = (L^)*,i^'i. V.? 6 G, / = 1, • • • , fc, where {Xi ■ ■ ■ , Xk) 
is the system of left-invariant vector fields in (13.21 ). 

Theorem 4.5. Assume y" : [0, 1] G be a strictly abnormal geodesic of the system (13.21 ) 
(we denote by y the curve (exp)~'-y in b) with'yiQ) = 1. Then there exists nonzero A* 6 b* 
such that 

(1) {a\ (F;)*,y(o [dy(t) exp)"' e,j = 0, V? e [0, 1], / = 1, • ■ • , k, where FJ : y(0 ^ y(t) 

is the Lipschitzflow determined by y(s) = (dy(.v) exp) y^{s), where y^ = n^{y), 

is the projection ofb onto the first layer V^; 

(2) the Goh condition holds: 

(4.7) I^A\ (d^(,) exp)"' [e,-, ej]^ = 0, V? G [0, 1], /, j = 1, • ■ • , k; 

(3) I^A*, (dy(,) exp)"' [[y\t), v] , vfj > 0, for a.e. ? 6 [0, 1] and Vv 6 V\ 

Proof. Applying Theorem |4.1| to the system (13.21 ). there exists nonzero 6 T'^^jG such 
that 2(0 = (P\)*A^ satisfies 

(4.8) <mx;(r(0)>=0,/= I,-- - 

and the conditions ([T]) and ^ in Theorem 14.11 , where A{t),P],Xi,Xj are replaced by 
A{t), P] ,Xi,Xj respectively. Here P^ : 7(0 ■y(r) is the Lipschitz flow determined 
by the system (13.21) . Keeping in mind y^ is the control, differentiating on both sides of 
7(0 = exp(y(0), using (|3.2I) we deduce 

(%i(o)*,ir^(0 = (exp)*,y(,)r(0 

which implies 

(4.9) 7(0 = (d^(o exp)"' y\t), V? 6 [0, 1]. 
Note that 

(4.10) = expoF;o(exp)-\ 

where FJ : y(0 yir) is the Lipschitz flow determined by (14.91) . Combining (14.51 ) with 
(133 we have 

i^;(r(0) = r(T) = (y(T),---,/(T)) 



where for i 
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(4.11) 



^'^^^ = Yj~;Jr Z J [r'^[r''^•■•[r''" ''r'^l'-'ll^+yW- 



Because is the control and is fixed when we consider the flow F], we deduce that 
FJ(y{t) + sv) = F]{y{t)) + sv when v e b verifies n^{v) = 0. Thus 



(4.12) 



(F;)^,^(,)=Idon \>\V\ 



Let ^0 = {Lyi^)T^^ 6 I?- Taking /I* = (d^(i) exp)* ^o, by (lOOl) and (I4l2l) we have for 
V?e [0,l],/,j= I,-- - 

= {I(0,K,X,](y(0)) 

= {(?;)*I\ [(L^(,))*,i^'Mfe))*,i^'i]) 
= {I\(Ly(f))^,i[ei,e^]) 

= (X\(P])^^(t){L-^(t))*,i{(^h ej\) 

= (exp)*,^(i)(F/)^,y(,)(exp);L^,)(L^(,))*,i[e,-, ey]) 

(4.13) = ^l', (exp)^,^(i)(F/)*,y(,) (d^(,) exp) ' [e,-, ey]J 

= ^/l\(exp)^,^(i)(dy(,)exp) ^ 
= ((exp)*(i)(L_^(i))*^o, (dyW exp) ' [ei,ey]J 
= ^(d^(i) exp)* ^0, (dy(f) exp) ' [e,-, ey]J 
= ^i*,(dy(,)exp) ^ 



where we used the explicit formula (|4.4I) for the definition of the differential of exp and the 
fact that {i^^yd) exp) ^ [e,, e^] j = by (|46l) . Similarly from (|4^ we obtain 



{i*,(F/)^,^(o(dy(oexp)-^e,} = 0. 
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The generalized Legendre condition ^ in Theorem 14.11 implies for i = I, - ■ ■ ,k, a.e t e 
[0, l]andv = (v\--- ,v'^)eR\ 

< (A(t), [[7{t)X(y(m] X(y(t))]) 

= {(PXA\ [[(exp)^,y(,)y(0, (%))*,iv] , (%))^,iv]} 

= (^)*,y(0 [[(eXp)^,^(;)7(0, (%(;))^,1V] , (Ly(t))ir,iv]) 

(4.14) ^ (^i, (exp)^,^(i)(F/)*,^(^)(exp);|-^,^ [[(exp)^,y(^)7(0, ' 
= ((exp)*(i)(L_y(i))*^o, [[r(0, (dy(o exp) ' v , (d^(,) exp) ' 
= ^^*,(d^(,)exp)"^[[y(0,v],v]^, 

where we used (|4.12l) . (|49l ). and v = ^i-i v'e,-. □ 

Remark 4.6. (1) From the formula (12.81) and (14.101) . one easily see that the condition ^ 
in Theorem 14.51 is just the abnormal condition in Camot groups. We call the system of 
equations 



(4.15) 



I^A*, (F/)*,y(,) (dy(,) exp) ' = 0, i=l,--- ,k. 



the abnormal equations in Camot groups. The map {Fl)i,^y(t) is complicated for Carnot 
groups with high steps, though we already know by (|4.12l) that it is the identity on b\V^. 
We give formulas of (F/)^ on Carnot groups up to 4 steps. Let y = (y^, - ■ ■ , y'^)- From 
(l4!6l) and (l49l) we have 

P]iyit)) = Y, r {-iy^{^AF]{y{t)))' y'(j)(H +y{t). 

Setting (FJ),,y^4ed =: ^[(t) = ^[.(t) + ■■■ + ^[/t), {^J. = nK^d 6 V^', j = 1, ■ • • , r) we 
obtain thus for t <t < I 



r-l ^ 
j=l 



M-_l)^ 

7"! 



j terms 



i^lAy,--- ,[y,y']---]] + --- + [y, iy,--- 



and thus 



(4.16) 



-B,J^[^l,y']cF= ^[ei,y\T)-y\t)], 
= ^ J k-, , 7^]] + Wei, y\y'W- ^[[e,-, r'(0], rVr) - y\t)]. 



+[^:2'[y,y]] + [r',[^:i,y]]^- 
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(2) The derivation of the Goh condition (|4.7I) in Theorem l4.5l is based on the graded struc- 
tures of Carnot groups. Similar computations can be carried on general nilpotent groups 
but the final formula may be more complicated than (14.71) in Theorem 14. 5 [ 

(3) By the same computation from (|4.3I) we can obtain 

(4.17) I^A\ (d^(,) exp)"' [y\t), [a, = 0, a.e. t e [0, 1], j = 1, ■ • ■ , k. 

Note that if G is of rank 2, then abnormal curves (not necessarily minimizing) satisfies the 
Goh condition |4]7] and llJTl see Remark |4]4l 

(4) Let G be a Carnot group of step 3, with Lie algebra b = V^®V^®V^ satisfying (I37T1) . 
Then the Goh condition (14.71) is just (see [38J) 

r A^ = o 

|i^[y(0,[a,Z7]] =0, Va,Z7 6 y^Vre [0, 1] ' 

where A* = iA\ A^, A^) 6 b\{0}, A' e (VT, i = 1,2,3. In fact, first letting t = Om (gjl) we 
get A^ = 0. Combing (14.61) with (14.71) . the desired statement follows from < A*, [e,, ej] + 
^[7\t),[ei,ej]] >= 0,ij= ,k. 



5. A REDUCTION ARGUMENT IN SUBRIEMANNIAN STRUCTURES OF STEP 3 

Liu-Sussmann [28] had given important results on abnormal geodesies of rank 2 distribu- 
tions. Assume A is a rank 2 distribution. Then /?a H T{A^)-^ is a line bundle on (A^)-^\(A^)-^, 

because /zA(<t) is transverse to T^(A^y in r^A"^ whenever O 6 (A^)-^\(A^)-^. If the Hamil- 
tonian lift Y = (y, A) of an abnormal curve y is regular (parameterized proportionally to 
arclength), i.e., T 6 (A^)-^\(A^)-^ is an integral curve of the line bundle, then y is locally 
optimal ( [,28. Theorem 5], see also |[T3] E1). We claim the converse is also true for rank 2 
distributions of step 3. For our purposes we first need the following slightly generalized 
fact, whose proof is essentially contained in [|2l Chapter 10]. 

Definition 5.1. An abnormal curve y in a subriemannian manifold (M, A, ^v^) is called a 
nice abnormal curve if its Hamiltonian lift T = (y, A) is a Lipschitz curve in (A^)"^\(A^)"^. 

Lemma 5.2. Let (M,A,gsr) be a subriemannian manifold of step r. We assume A can be 
spanned by a system of orthonormal vector fields {Xi,X2, Ti, - ■ ■ , T,„} satisfying 

(5.1) [Ti, A'] c A' for i = 1, • ■ • , m and s = 1, ■ • ■ ,r. 

If a nice abnormal curve (parameterized proportionally to arclength) y : [0, 1] ^ M 
satisfies the generalized Legendre condition 

{A{t), mt\Uy{t))lXi{ym) > 0, fora.e. t e [0, 1],/ = 1,2, 

and 

(5.2) y(0 = u\t)X,(y(t)) + u\t)X2iym a.e. t e [0, 1]. 

Then its Hamiltonian lift T = (y. A) is an integral curve of a smooth vector field on 
(A^)"'"\(A^)"'". In particular, ify is a nice abnormal minimizer (parameterized proportionally 
to arclength) satisfying (15.21) . it is smooth. 
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Proof. First the condition (15.11) implies = span{X, [Xi,X2]} and = span{X, [Xi,X2], 
[Zi, [Xi,X2]], [Z2, [Xi,X2]]]}. From the assumptions on 7, the Hamiltonian lift T = (y, /I) 
is a Lipschitz curve in (A^)"^\(A^)"^ with 

Y(0 = u\t)X\{i:{t)) + u^{t)Xl{l'{t)) a.e. ? e [0, 1]. 

By Lemma I23] 

u'iDhni'^it)) + uHt)h2nm)) = a.e. t e [0, 1], 

where /Z2I2 = /?[X2,[x,,X2]],/2ii2 = kxaxuX2]]- Because T ^ (A^)^, (/iii2(T(0))'+(/j2i2(T(0))' ^ 
0, V? e [0, 1]. Since iu\t)f + {u\t)f = C > a.e. f e [0, 1], we get 



{u\t)f 



C{h2nm))f 



(/?112(T(0))' + (/^212(T(0))' 

and 

iuHt)f= C(.n2(T(0))^ 



(/l„2(T(0))' + (/?212(T(0))' 

So it follows a.e. t e [0, 1], 

1,,, VC/i2i2(Ta)) 2,,, -VC/in2(T(0) 

(5.3) M (0 = - ■ u (t) = 



V(^ii2(T(0))' + (hinim)? V(^ii2(T(0))' + (/?2i2(T(0)) 



2 



or 



,.4, 1,,, -Vc/i2i2(T(0) 2,,, Vc/in2(T(0) 

(5.4) u (t) = . u (t) = 



V(/^112(T(0))' + (/J212(T(0))' V(/^112(T(0))' + (/^212(T(0))' 

Assume there exists a subset A of [0, 1] such that u^{t),u^{t) are valued as in (15.31) a.e 
t e A while valued as in (15.41) for a.e. ? 6 [0, 1]\A. For our case the generalized Legendre 
condition reads 

< {A{t), [[r(0,^i(r(0)] ,^i(r(0)]> = u\t)hn2m)) a.e. t € [0, 1], 

and 

< {A{t), [[r(0,^2(r(0)] ,^2(r(0)]> = -MkO/^2i2(T(0) a.e. r G [0, 1], 
which implies that 

(/^ii2(T(0))' < and (/i2i2(T(0))' < a.e. t e A. 

Because (hmC^it)))^ + (hiuC^it))? ?^ 0, V? e [0, 1], A is a null subset. So u\t),u^{t) are 
valued as in (|5.4I) a.e. t e [0, 1] and u^,u^ (or after a modification on a null set) are actually 
Lipschitz continuous. Thus 

t(o = -y(T(0), V? 6 [0, 1] 

where 

y(0) = — \ — '- 

is a smooth vector field on (A^)-^\(A^)-^ (a smooth submanifold of A-^). □ 
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Remark 5.3. (1) The statement of Lemma [S!2] had been made in [|28l Corollary 4] for rank 
2 case. But Proposition 10 there did not consider the following cusp type singularities: 

V(Y(0), te[0,tol 
-y(T(0), teitoM 



(5.5) T(0 



for ?o e (0, 1). Such T is simple with a cusp at to and a priori one did not know its optimality 
(of course, the cusp type singularities may be excluded by a blow up argument as in [[27|. 
see Theorem 13. 21) . 

(2) For the rank 2 of step 3 case, by Theorem 14.11 the Hamiltonian lift Y = (y. A) of a 
strictly abnormal minimizer y is automatically not in (A^)"^, so in this case the optimality of 
y is a sufficient and necessary condition for the fact that its Hamiltonian lift T is an integral 
curve of a smooth vector field. 

Proposition 5.4. Let (M, A, gsr) be a subriemannian manifold of step r, with A locally 
spanned by an orthonormal basis {X\, ■ ■ ■ ,Xt, Ti, - ■ ■ , T^} where 

(5.6) A-n c A•^ /or / = 1, • • ■ , m, ^ = 1, • ■ • , r. 

If a strictly abnormal minimizer j {parameterized proportionally to arclength) is nice and 
satisfies 

k 

(5.1) r(0 = J] u\t)Xj(r(t)) a.e.te [0, 1], 

j=i 

then the Hamiltonian lift T = (y. A) of y satisfies the following properties: for any to e 
[0, 1], there exists an interval /q c [0, 1] containing to, such that Y|/„ is also a length mini- 
mizer in a subriemannian manifold (Mo, A,^sr) c (T*M, A'^, g*^), where Mq is an immersed 
submanifold of T*M, A = spanjXi , • • • , X^-i , Ti , • • • , Tn}, X, 's are linear combination of 
X':'s, satisfying [Tj, A^] c A', for j = 1, • • • ,n, s = 1, • • • ,T and T/^ verifies 

k-\ 

(5.8) t(0 = Yj^\t)Xj(y(t)) a.e.te h. 

;=i 

Proof. Let y be a nice, strictly abnormal minimizer on (M, A, g^^) satisfying (|5.7I) . By the 
niceness condition and Theorem 14. 1[ the Hamiltonian lift T = (y, A) with 

k 

(5.9) t(0 = uXt)X':{r{t)), a.e. t e [0, 1] 

1=1 

is a Lipschitz curve in (A^)-^\(A^)-^. By Proposition I2.9[ Y is also a length-minimizer 
in {T*M, A', gt) where A^ = span{X[, • • • , XI, f{, • • • , f^, , • • • , T,,}. Here {T,,--- ,T„] 
spans the vertical bundle of T*M, and ^* is defined by declaring the basis orthonormal, 
see Section 121 Note 



{A'f = A' + span i,j=\,--- ,k] 



and 



(A^)-^ = {A'f + span [{X^, Xj]], i, j, s = I,- ■ ■ ,k] , 
since [r,, Tj\ = 0, [r,,Xn = /r,, [f = [Tj,X,Y 6 A^ 
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Because T e (A^)-^ we have (see Lemma l23l) 

k 

(5.10) Yu "'(OV.K,z,]](T(0) = 0, a.e. t e [0, 1],/,; =l,---,k. 

Let ^0 G [0, 1]. From T(?o) i (A^)"^, there exists so,io,jo (without restriction, say sq = 
l,zo = 1, Jo = 2) such that /?[Xi,[Xi,X2]](*^(^)) ^ 0- By the continuity of Y, we may assume 
^[x,,[Xi,X2]](T(0) * for any t e Iq := [to-eo, ?o + 6o] (for to = 0, 1, take /(, := [0, eo], [l-6o, 1] 
respectively). 6o > is chosen so small that all frames involved are well defined in a 
neighborhood ^ of ^{7,^) and /j[x,,[Xi,X2]](<I>) ^ 0, VO 6 ^. So from (15.101) we have 

k , 

(5.11) u\t) = y uXt)-^irm a.e. t e k 



r=2 



112 



where we set /zn2 = \xiXXxX7\\-' ' = 1' ' " ' ' ^- Putting (15.111 ) into (15.91) . we obtain 
t(0 = "'(^) (^^? + ("^(0)' a.e. t 6 /o. 



Note that + ^i^}._2 linearly independent in G . By a process of Gram-Schmidt 

orthonormalization, the last formula can be written as 

k 

(5.12) Y(0 = _^7?(0X,(Y(0), a.e. t e /q, 

(=2 

where X,(/ = 2 • • • , A:) (defined in is expressed as a linear combination of • • • 
Set A = span{Z2, • • • , X^., fj, • • • , f^, Ti, • • • , r„}. By the orbit theorem of Stefan and Suss- 
mann [|35l[371 . there is an orbit Mq of A|^ through T/^. We still denote by A the restriction 
on Mo of A. Define 'g^r by declaring {X2, ■ • ■ , X^, f [, • ■ • , f^, T\, - ■ ■ , r„} an orthonormal 
basis of A. Then (Mo, A, g^^^) is a subriemannian manifold satisfying (15.61) . It is clear that T/^ 
is a length minimizer of (Mq, A,'^^,,-), because (Mq, A,^^,.) is a sub-system of {T*M, A'^,gsr) 
(noting that 'g^r is just the restriction to A of g^r)- □ 

The reduction process in Proposition 15.41 could be continued if the curve Y/^ is still nice 
in (Mq, a, ^5,.) (of course, if Y/^ is normal in the new subriemannian manifold, we are done). 
But in general the resulted distribution A is complicated: we even have no information on 
its growth vector. In the above proof we easily see that the restriction on Mo of A* = 7r*A 
may annihilate A^ at Y(?o) (A* certainly vanishes on A^ by Theorem 14.31) . That is, the 
restriction on Mo of A* may not be the Hamiltonian lift of Y/^ when Y/^ is an abnormal 
minimizer in (Mq, A,'gsr)- In the rest of this section we concentrate on a special class of 
subriemannian manifolds. 

Definition 5.5. We call a subriemannian manifold (M, A, ^^r) of step r is nilpotent, if there 
exists a local basis (not necessarily orthonormal) {Xi, ■ ■ ■ ,Xi,} of A such that all r -I- 1 step 
commutators of X,'s are vanishing. We call {Xi, ■ • ■ , Xj,} is a nilpotent basis of A. 

Theorem 5.6. Let (M,A,gs,.) be a nilpotent subriemannian manifold of step 3. Then any 
strictly abnormal minimizer is smooth. 
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Proof. First from the proof (see e.g. [jlSllMlD of the lifting Lemma [34l the lifted vector 
fields X,'s in (|3.5I) is also nilpotent of step 3 if X,'s is nilpotent of step 3. So by Lemma [341 
we may assume [Xi, ■ ■ ■ , Xt) is a nilpotent basis free up to step 3. Let 

= ■ ■ ■ ,Xk,Xj^+i, ■ ■ ■ • • • ,X,j} 

be an adapted frame such that {Xi , • • • ,Xk, Xk+i, • • • , X„^ } spans . Note that for some fixed 
Y G {Xk+i, • • • ,Xn2}, we may further assume for / = 1, • • • , A:, [X,-, Y] e {X„2+i, • • • ,X„}. Let 
^ = {a»\ • • • , o)"} be the dual coframe of £/ and ^ be the adapted basis for the vertical 
bundle 1^(7* M), see Definition^ 

Let 7 be a strictly abnormal minimizer on (M, A, g^r) satisfying 

k 

m = J]u\t)Xj(y(t)) a. e. re [0,1]. 

7=1 

By Theorem 14. 11 the Hamiltonian lift T = (y, A) with 

k 

(5.13) m = u'(t)X^(r(t)), a.e. te[0,l] 

i=l 

is a Lipschitz curve in (A^)-^\(A^)-^. By Proposition 12.91 T is also a length-minimizer 
in {T*M, A'^g*,) where A^ = span{X;\ • • • Ti, • • • , r„}. Here {Ti, ■ ■ ■ ,T„] spans the 
vertical bundle of T*M, and g*^ is defined by ^*(X[,Xp = g,r(Xi,Xj),g*XTs, T,) = 6'' for 
i,j= 1, • ■ • ,k, s,v = 1, • ■ • , n, see Section[2l Note that 

(A^f = A'^ + span {[X':,X'jl ij=l,--- ,k] 

and 

(A'^)^ = (A'f + span {[XJ', /, j, s = I,- ■ ■ ,k} , 

since [T,-, Ty] = 0, [X', T,] = (LxJy, VX 6 jr(M). 
Because Y e (A^)-^ we have (see Lemma [231) 

k 

(5.14) "'(0/^k.k,z,]](T(0) = 0, a.e. t e [0, 1], /, j = 1, • • • , fc. 

Let ^0 6 [O5 1]- From T(to) i (A-^)-^, there exists i'cz'oj Jo (without restriction, say sq = 
l,zo = 1, Jo = 2) such that /z[Xi,[Xi,X2]](*^(^)) ^ 0- By the continuity of Y, we may assume 
^[x,,[Xi,X2]](T(0) ?i Oforanyr 6 /q := [?o-6o, ^0 + ^0] (for?o = 0, l,take/o := [0, eo], [1-eo, 1] 
respectively). 60 > is chosen so small that all frames involved are well defined in a 
neighborhood & of t{Tt^) and /z[x,,[Xi,X2]](<I>) ^ 0, VO e ff. So from (15.141) we have 

k ^ 

(5.15) iiHO = y u\t)-^{T{t)\ a.e. ? e /q 

^ ^112 

where we set /zai = ^k,[Xi,X2]]5 ^ = 1^ ■ • ■ ^k. Putting (|5.15l) into (|5.13|) . we obtain 

(5.16) Y(0 = _^7?(0X,(Y(0), a.e. ? 6 /q, 

where X,- = /zniXj + /zi^X^, ■^'(O = i = 2--- ,k. Note that {X2, • • • ,X^} is linearly 

independent in and also free up to step 3. Set A = span{X2, • • ■ , X^, Ti, - ■ ■ , T^J. We may 
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assume X„2+, = [X,-, [Xi, X2]], i = 1, • ■ • ,k. Since ^ is the dual basis of hmix. A) =< 
A,Xr„+i(x) >= ^n2+iix) for A = 2"=i ^Mi^) and thus 

(5.17) T,ha2 =0,s = 1, • • • ,n2,i = 1, - • • 
So we have for / = 2, • ■ • , k, 

[T„Xi] = [T.^hniXl + hniX'r] = hmiXl, T,] + hniiX';, T,]. 
From the nilpotency of X/s and Lemma [Z2] we have for / = 1, • • • ,k,s = 1, • • • ,n2, 

«2 

(5.18) [X% T,] = J] (<^\ iXs,Xi]) T„ 

s=l 

from which we obtain [r„X,] 6 A and for /, 7 = 1, • • • ,k,s= 1, • • • , ^2, 

"2 



(5.19) 



[X;, [Xl T,]] = ((^-^ [Xs,Xi]) [X-j, T,] 

= J] {co\[X„Xi])(ajMX,,,Xj])T,. = 0. 



s,s'=l 

Using (|5.17l) - (|5.19l ) we get for 5, 5' = 1 , ■ • ■ , n2, i, j, v = 2, • ■ • ,k, 

[ry,r,] =0, 

[Xi,Xj] = [hiuX'i + hniXlhjnXl + hmX'^ 

= h],2[XUX'j] + hn2hniVX\,r^ - hnihjnVXlXl 
[Xj, [T„XiW = [hjuXl + hu2X'j,hn2[Xl, T,] + hniiXU T,]] = 0, 
[T,,[T„Xi]]=0, 

[X,, [Xi,Xj]] = Knh\,2VX\, VX%rj]\ + KuhnhniVXl, VX\XjW 
- KnhuihjniXl + hl,^[X'^„ [XlX'j]] 

+ hinh\,2{K, {X\,TjW - h^uihjniX:, [XJ,^^], 

where we also exploit the nilpotency of the system {Xi, ■ • ■ ,Xf-}: 

X'jhiu = X'jh[Xi,[Xi,X2]] = h[Xj.[Xi,[Xi,X2]]] =0,i = 1, • ■ • ,k and j = 2, • ■ • ,k. 

Thus all 4 step commutators of X;'s and T/s in A are vanishing. 

On the other hand, by the orbit theorem of Stefan and Sussmann [|35l l37l . there is an 
orbit Mo of A|^ through Tj^. We still denote by A the restriction on Mq of A. Let^„. be the 
restriction on A of ^* . Then (Mq, A,^^,-) is a nilpotent subriemannian manifold of step 3. It 
is clear that T/^ is a length minimizer of (Mq, A,^,^), because (Mq, A,'g„-) is a sub-system of 
(T*M, A'^,gsr)- If T/q is a strictly abnormal geodesic in (Mq, A,^,,.)^ we continue the above 
process. In each step, we can check that the new subriemannian manifold is of step 3 and 
nilpotent. After at most k - 2 steps we will arrive at the rank 2 case. By Lemma [5]2l T is 
smooth near tQ. By the arbitrariness of to, so y is smooth. □ 
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6. Abnormal geodesics in some Carnot groups of rank 2 

As pointed out in Section [31 the smoothness of abnormal geodesics in Carnot groups 
will shed some light on the regularity of geodesics of general subriemannian manifolds 
through a blow up argument (see Theorem 13.21) . In this section we concentrate on Carnot 
groups of rank 2 and get some results from the Goh condition. We proved in [|38]| that all 
subriemannian geodesics in Camor groups of step 3 are normal. In the rank 2 case, the Goh 
condition is more convenient although it is essentially implied by the abnormal condition, 
while the latter is actually an integral-differential condition (not an algebraic-differential 
equation as stated in some references) if the step of the Carnot group is bigger than 2, see 
Remark 14^61 and below. 

First we need a lemma about horizontal lines through the origin. 

Lemma 6.1. Let (G', A, g^r) be a rank k Carnot group of step r. Let'y : [0, 1] G' be an 
integral curve of a horizontal left-invariant vector field with 'y{0) = 1. Then -y is a normal 
minimizer. 

Proof. Let [X^, ■ ■ ■ ,X,t} be horizontal left- invariant vector fields generating the Lie algebra 
b' of G'. Identifying G' with b' = T^G' by the exponential mapping, y satisfies for t e [0, 1] 

k 

(6.1) ^{t) = Y,c%m)\ 

i=l 

where c\ • ■ • , c*^ are constants. By Lemma [TTl y is regarded as a curve y in b which is 
the horizontal lift of y\t) = 2i=i c'tei, where {ei, • • • , et} is the ^-generators of b satisfying 
Xi(q) = {Lq)i,^iei, i = 1, • • • ,k,'iq e G'. Extend the system of {Xi, • • • ,Xk} {{ei, • • • , ek}) to 
be a left-invariant basis {Xi, - ■ ■ (a basis {ei, • • • ,e„}) of TG' (of b'), and also extend 
gsr to be a metric ^ on G by declaring {Xi, - ■ ■ orthonormal. So b' = R" under this 
basis and G' = R" with a group product where X,- can be written as Xj = 'Yj"i=\^lj^ where 
Xj are homogenous polynomials. Then the projection P : R" 9 (;c\ ■ • ■ , x'', ■ • ■ , x") 
{x^ ,■ ■ ■ ,)^) 6 R*" satisfying P(y) = y^ (y is the horizontal lift of y ^ with y(0) = 0) is 
a Riemannian submersion from (R",g) to (R'', < ■, ■ >), where < •, ■ > is the standard 
Euclidean metric. Since y^{t) = (fc\ • • • , tc^) is shortest in (R*", < •, • >), y" (just y' itself 
seen in R") is shortest in (R", g) and thus also is a minimizer in (R", A, g^r)- 
Since y is shortest in (R", g), there exists a Hamiltonian lift (x, X) such that 



(6.2) 



dHr _ 

i- 

--J]{A,X^(x)){A,D,X^(x)) 



x=-^ = Y,{A,Xt(x))Xt{x) 

(=1 

dHr 



A = - 

(ix 

i=i 



where HXx,A) = ^ i'^^^ti^yf' is the Riemannian Hamiltonian of (R",^). Note that in 
(lOl) . {A,Xi{x)) = for / = A: + 1, • • • ,n and {A,Xi{x)) = c' for / = 1, • • • by (O. So 
{x. A) actually satisfies the subriemannian Hamiltonian equation, recalling the subrieman- 
nian Hamiltonian of (R", A, g^r) is Hr{x, A) = \ Jiti '^ii^yf', see Remark [Z6l □ 

6.1. The case of step 4. By Lemma [341 the study of abnormal geodesics in any rank k 
Carnot group of step r can be reduced to the study in a free Carnot group with bi-dimension 
{k, r). So in this subsection let G be a free Carnot group with bi-dimension (2, 4). Let b be 
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its Lie algebra, b = V^ey^eV^eV"^ where = spm{ei,e2},V^ = span{^ = [^1,62]},^^ = 
span{[ei, [62, ?]} mdV"^ = span{[ei, [e^?]], [^2, ^2, ?]], [ei, [ei, ?]]}• We already knew 
that a horizontal curve 7 in G can be regarded as a curve y = + + + y"^ e b which is 
determined by the first layer y^ via the equation (I4.9I ). 

Theorem 6.2. Ify : [0, 1] ^ b is an abnormal minimizer (parameterized proportionally to 
arclength) in G with y{0) = 0, then y must be a line in or its interval through the origin. 
So G admits no strictly abnormal minimizers. 

Proof. Since y is an abnormal curve in G which is of rank 2, from Theorem 14.5 l and Remark 
14.61 there exists nonzero A* = Ai + A2 + At, + A4 with At e (V)*, i = 1, 2, 3, 4, such that y 
satisfies the Goh condition (14.71) : for any t e [0, 1] 



= ^/l*,(d^(,)exp) 



[ ^ Id + ^ad(r(0) + ^ (adr(O)' } ^ ) 



(6.3) = ^'^*,^ + ^[r(0,?] + ^[r(0,[r(0,?]]| 

= ? + \ir\t) + y\t), + j^[rkO, [r\t), ?]]| 

where we used the facts that b is nilpotent up to step 4 and dim(y^) = 1 (thus [^1,^2] = 
for any ^1, ft e V^). In (16.31) let r = 0, we get A2 = and thus (16.31) becomes 

(6.4) + ^4, ^[y^O, + ^[r'(0, [7\tl ?]]| = 0, V? 6 [0, 1]. 

Note that y also satisfies (14.171) (see Remark |431) : for a.e. t e [0, 1] 

= (i*,(d,(,)exp)"'[ri(0,?]J 

= /r,(ld + ^ad(r(0)) [Vit),?]] 



(6.5) , , > 

= a*,[y(o,?] + -[m[y(o,?]]) 

^3 + ^4, (0, + ^ (0, [rHO, ?]]) . 



2 

u 

2 

From the Jacobi identity we have a.e. t e [0, 1] 

^ [r'(0,[7\0,d] = [r\0,[y(0,?]] + [r'(0,[y(0,d] 
(6-6) = [[y\t),r\t)],?] + 2[y\t),[7\t),g]] 

= 2[y\t),['y\t),g]]. 
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Combing ([63]) and ^M, it follows {y\0) = 0) 

+ ^4, [7^0,?] + \[7\t),[7\t),g]]^ = 0,V? 6 [0, 1], 

which together with (16.41 ) implies that 

(6.7) {A,,[7\t),g]) = 0,-ite[0,l] 
and 

(6.8) (^4, [7\t), [7\t), g]]) = 0, V? 6 [0, 1]. 
(♦) If At, + 0, the condition (16.71) can be written as 

<i3,[ei,d > < /t3,[e2,?] > Jc^(0 = 0,Vf6 [0, 1], 

where = jc^^i + x^e2. Thus jc2(0 = c^x\f),'it e [0, 1] or x^t) = c^x\t),\/t e [0, 1] for 
some constants c^,c^. Because we assume y is parameterized proportionally to arclength, 
we get that and thus y are an interval of a line in through the origin. 

(♦♦) If /I3 = 0, then A4 + 0. Otherwise we also have A\ = ^ and thus A* = (a 
contradiction). In fact, if /I4 = 0, from (14.151) and (14.161) we have for e,, i= 1,2 

= (a" , (Fj)i,,yit) (dy(,) exp) ' 
= I^A*, {F])^^y(t)ei + (dy(,) exp) ' e,- - 
= {a\ + ^,12 + + ^l, + (d,(, exp)'' - 
= {Ai,ei) + I^A2 + A3+ A4, + ^l-i + + (dy(f) exp) ' e,- - e,j 

where we used the facts that {F])i,^y(t) is the identity on t)\y' and {dy(,) exp) e,- - e,- ^ 
Thus A^ i^O and the condition (16.81) can be written as 

(6.9) a{x^f{t) + 2bx\t)x^{t) + c(;c2)2(0 = 0, 6 [0, 1], 

where a =< /I4, [^i, [^i, ?]] >,b =< A4,[eu[e2,g]] >, c =< /I4, [^2, [^2, ?]] >• Without 
restriction we assume a > 0, then (16.91 ) has no solutions if < ac, or 

, ^ -b+ VZ72 _ 
a 

if b^ - ac > 0. For the latter case, y as a length minimizer (parameterized proportionally to 
arclength) must be an interval of a line in through the origin, because y has no comer 
type singularities, see [|77l P. 581] or Corollary 16.41 

Our statement follows from Lemma [6?T1 □ 

Theorem 16.21 implies that each rank 2 Camot group of step 4 has no strictly abnormal 
minimizers by Lemma 13.41 and Lemma 16. 1[ Our result contradicts with the statement in 
Gole-Karidi [21 J where they claimed a strictly abnormal minimizer in a rank 2 Camot 
group of step 4. We clarify here that the curve they found is not an abnormal extremal. 
They used a wrong system of abnormal equations (they exploited another equivalent char- 
acterization for abnormal extremals, see Remark [2. 101) . Instead of directly checking that 
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the curve does not satisfy the abnormal equation (|4.15l) . we point out that the error appear- 
ing in the derivation of abnormal equations there is at line 7 of Page 539. In the setting of 
moving frames for r*G, the derivative of a curve in r*G is not componentwise in general. 

To be more precise, given a system of left-invariant vector fields {Xi , • ■ • , Xj,, ■ ■ ■ ,X„} of 
a Camot group G such that {Xi, ■ ■ ■ , X^} is the /:-generators of its Lie algebra. We use the 
same notations as in Definition 12. II to denote the dual coframe • ■ • , oj"} of T*M. The 
identification of G with its Lie algebra b = TiG gives a natural coordinate system x^, - ■ ■ ,x'^ 
in G: 

G 9 x < — > {x^ ,• ■ ■ ,^") <=^ X = exp(x'ei -I- h x"en). 

Assume G is of step r and {Xi, ■ ■ ■ • • ■ ,X„.} is a basis for A', / = 1, ■ • ■ ,r. Then 

under the natural coordinate system 

where X^^_^j = 0, for 5 = 1, • • • , n,- -h j - 1, X^'l^j = 1 and X^^,_^j for s = rii + j + \,- ■■ ,n 
is a (ojs - ?')-homogenous polynomial. That is, (Xf ) is an upper triangular matrix (rows 
indexed by s and columns by s') with diagonal terms equivalent to 1. For A = 2"=i -^sdx^ = 
Yjs=i ^sOJ'\ the natural coordinate (/li, • ■ • , An) and the adapted coordinate (^i, ■ • ■ ,^„) are 
related by = Z"=i^f^.' and thus ^ = Z"'=iB-l,-^,s = ,n, B = (B^,) is the 

inverse of the matrix (Xf ), that is, 2"=i 5* = Z"=i B'jXi = 6^ for z, ^ = 1, • ■ • ,n. So 
B is also an upper triangular matrix with 1 as diagonal terms (other nonzero terms are 
homogenous polynomials). Now given an abnormal extremal O = (y. A) in r*G, denoted 
by {x(t),A{t)) and (x{t),^(t)) in the above two coordinate systems. The abnormalness of O 
implies ^i(t) = • • • = ^<-(f) = 0,'it. In the natural coordinate system the derivative of A is 
componentwise, that is, for a.e. t 



d d 

.v=l ^ s=l 



( n 



Vv=l /.v'=l 



d 



= j]j]B:(x(t))x:,ixit))Ut)-^ + J] J] j{B:{xmxi,{x{t))ut)^ 

v=k+\ S'=\ 1=1 v=k+\ s,s'=\ 

= X ^'^^"S Z Z B:(x(t))-ix:,(xmm 



v=/t+l .V'=l V=i+1 .V=l' + 1 

from which we see that if the step r of G is bigger than 2, then the second term in the last 
formula is not vanishing in general. In fact, for example if ni < s' < n2, then the coefficient 
before 

" ^' d 

v=n2 s=n2 

is typically non- vanishing, because both 5^ and Z^, are at least 1 -homogenous polynomials 
of x(t). 

6.2. A note on corner-type singularities. In this subsection, we study the optimality of 
the curve of (13.41) when it has a comer. Leonardi-Monti [[27ll using a shortening technique 
(such technique also appeared in [28, Appendix E]) proved that the curve is not minimizing 
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under some conditions. We will use the system of abnormal equations (14.151) to get non- 
optimality in some cases. 

Assume ? = is a corner of the curve, that is, 1^+, -U- are linearly independent. By a 
left-translation and re-parameterization, we change the curve 7 to 7 which is the horizontal 
lift of 7': 

[ te2, 1 6 [0, 1-] 

(6.10) 7^0 = 1 1 1 , 

y{t - -)ei + -62, 

where €1,62 are two linearly independent directions in . Thus 7 lies in the subalgebra b 
generated by {61,62}- If -y is length-minimizing, then 7 is also minimizing in the subgroup 
exp b. By Lemma 13.41 without restriction we study the optimality of 7 in a free Camot 
group with bi-dimension (2, r). 

Theorem 6.3. Lef (G, A, g^r) b6 afr66 Camot group with bi-dim6nsion (2, r). Th6 horizon- 
tal lift 7 0/7^ is a r6gular curv6 if and only ifr<6. 

Proof. Let b = ® ■ ■ ■ ® V'' he a free Carnot algebra with bi-dimension (2, 7). We want 
to prove that if the horizontal lift 7 = (y^, - ■ ■ , 7^) of 7' satisfies the system of abnormal 
equations of (14.151) for A* = ej^^^,- e b* with Ai e (V')*, then ^1 = ■ • ■ = ^6 = 0, but Aj 
may be nonzero. To this end, we need compute 

Pi(t) = (Fj)^^y^t) (dy(o exp) ^ 6i = ^l(t) + (dyf^t) exp) ' e,- - e,-, z = 1, 2. 

To simplify the notation, set 

ei,i = ei,6i2 = e2,62,i = \6\,e'A,6^x = Ve\,e2,\\,e^2 = [(^2, £2,1], 

^4,1 = [ei,63j],64,2 = [e2,e3,i],e4,3 = [£2,^3,2], 

^5,1 = [ei,e4,i],e5,2 = [^2,1, £3,1, ^5,3 = [^2,1, £3,2], ^5,4 = [^2,^4,1], 

^5,5 = [e2,e4,2],e5,6 = [^2, £4,3]' ^^6,1 = [^1, ^sj], ^6,2 = [^2,^54], 

^6,3 = [^2,65,4], £6,4 = [£2,^5,5], £6,5 = [£2,65,6], £6,6 = [£3,1,£3,2], 

£6,7 = [£2,1, £4,1], £6,8 = [£2,1, £4,2], £6,9 = [£2,1, £4,3], 

^7,1 = [£i, £6,1], £7,2 = [£3,1, £4,1], £7,3 = [£3,1, £4,2], 

£7,4 = [£3,1, £4,3], £7,5 = [£2, £6,1], £7,6 = [£2, £6,2], 

£7,7 = [£2, £6,3], £7,8 = [£2, £6,4], £7,9 = [£2, £6,5], 

£7,10 = [£2,1, £5,2], £7,11 = [£2,1, £5,3], £7,12 = [£2,1, £5,1], 

£7,13 = [£2,1, £5,4], £7,14 = [£2,1, £5,5], £7,15 = [£2,1, £5,6], 

£7,16 = [£3,2, £4,1], £7,17 = [£3,2, £4,2], £7,18 = [£3,2, £4,3]- 

That is, {6s J, s = 1, ■ • ■ ,1 , j = 1, • ■ • , dim V^} is a Hall basis of b. 
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First for any t 6 [0, j], y\t) = ■■■ = y^(t) = 0. For t e [j, 1], by KWi we have 
7 (t) = (t - -)ei + -62 

7 1 1 

r(t) = -^(t - -)e2,i 

r\t) = -1/2304^- l)e5,6 - 1/2304^- 1)2^5,3 - l/516(t - ^fe^j 

- l/2304it-^fe5,5 - l/S64(t-^fe5,4 - 1/864^-1)^^5,2, 

y\t) = -l/7680a- 1/2)66,5 + l/n520it-l/2f66,9 - 13/46080a- l/2)^e6,4 
+ 1/69 12a - 1/2)^^6,6 - 7/23040(? - 1/2)^6,2 - 7/34560(r - 1/2)^^6,3 

- l/3200(f- 1/2)^66,1 +37/69120(?- 1/2)^667 

+ l/4320(f- 1/2)3^6,8. 

Let P^(t) := n'(Pi{t)), s = 1, • • • , 6, / = 1,2. Let Using and (I4l6l) we get for t e 
[1/2,1] 

(6.11) P\(t) = 6,j, 

(6.12) P](t) = -l/462,u 

(6.13) Pl(t) = (1/12 - 1/240^3,1 - 1/48^3,2, 

(6.14) P^^it) = (1/96 - 1/960^4,1 + (1/48? - 1/96)^4,2 

(6.15) P^it) = (1/1920- 1/1 152/'2)e5,4 + l/2280(-10/'^ - 7? + 15?^ + 1)^5,1 

+ (-1/576 + 1/192? -5/1152?^)e5,2 + (-1/11520+ 1/14400^5,5 
+ (7/11520- 1/9600^5,3 + 1/11520^5,6, 

(6.16) P\it) = (-11/138240 - 7/34560?^ + 7/23040?^ - 1/230400^6,2 

+ (-17/92160- 1/1 152?^ + 1/2880?^ + 1/4608?+ l/1920?>6,i 

+ (-1/1 1520?^ + 1/11520?- 1/23040)^6,8 

+ (-7/138240 + 41/34560?^ + 13/23040?- l/640?2)e6,7 

+ l/2304(? - 1/2)2^6,6 + l/23040(-?2 + ? - 1/2)^6,3 

+ (-1/11520?+ 1/23040)^6,9 + (-1/23040?+ 1/46080)66,4, 

(6.17) P](t) = (1/276480?^ - 1/552960)^7,4 

+ (1/69120?2 + 13/1105920 -7/276480?)e7,i4 

+ (1/138240?^+ 1/221184- l/92160?)e7,ii 

+ (-13/1658880 - 1/138240?^ - 1/138240?^ + l/46080?)e7,io 

+ (-1/69120- 1/6912?^ + 1/3456?^ - 1/4608?^ + 11/1382400^7,2 
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+ (7/829440 - 59/414720?^ + 59/276480/'2 - 13/1382400^7,3 

+ (-13/829440 + 23/414720?^ - 23/276480?^ + 7/1382400^7,16 

+ (-7/552960 - 1/23040?^ + 1/230400^7,17 

+ 1/3317760(103 + 56t^ - 160?^ + 204t^ - 2000^7,12 

+ (23/1658880 - 1/46080?^ + 1/23040?^ - 1/276480^7,13 

+ (-23/1935360^ + ll/3870720)e745 + (-1/430080 + l/276480r>7,7 

+ (601/29030400 - 7/23040?^ - 59/69120?^ + 83/103680?^ 

+ 59/172800?^ + 1/1843200^7,1 + (1/645120? - 13/3870720)^7,18 

+ l/23224320(-173 + 1736?^ - 3136?^ + 1764?^ - 560^7,5 

+ (-1/430080 + 1/138240? + 1/138240?^ - 1/92160^^7,6 

+ (-1/1935360?- 1/430080)^7,8 - 1/1935360^7,9 

and 

(6.18) Pl(t) = e,,2, 

(6.19) P^(0 = (-3/4 + 0^2,1, 

(6.20) Pl(t) = 1/48(7 - 18? + 12?2)e3,i + (-1/6 + l/4?)e3,2, 

(6.21) P^it) = l/48(-l +4? + 2?^ -5?V4,i + 1/96(3 - 5? + 2?V4,2 + 1/96(2? - 1)^4,3, 

(6.22) Plit) = (17/576?^ + 1/144?^ -7/288?^ + 29/11520- 17/1 152?)e5,i 

+ 1/2880^5,6 + (-73/34560 + 5/1728?^ - 1/128?^ + 1/1440^5,4 

+ (-1/5760 + 7/1152?- 1/I28?2)e5,5 +(V/6912- 19/1728?^ - 1/96? 

+ 23/1152?V5,2 + (-1/384 + 7/1152? - 5/1 152?2)e5,3, 

(6.23) Pl(t) = (1/768?^ - 1/512?2 + 1/1536? + 1/9216)^6.6 

+ (107/34560? - 167/23040?^ - 1/2880 + 73/11520?^ - 31/17280?V6,7 

+ (-1/17280?^ + 11/11520?^ - 7/4608? + 23/34560)^6.8 

+ (-1/10240 + 1/7680?^ - l/23040?)e6,9 + (1/5760? - 1/11520)^6,5 

+ (1/480? - 71/11520?^ - 1/3840 + 49/5760?^ - 1/180?^ + l/720?^)e6,i 

+ (91/276480 + 13/8640?^ - 131/69120? + 23/5760?^ - 67/17280?^)e6,2 

+ (-1/8640 - 23/23040?^ + 23/34560?^ + 1/I920?)e6.3 

+ (-11/92160 - 1/4608?^ + 1/38400^6,4, 

(6.24) Plit) = (-1/23040?^ + 1/51840 + 1/25920?^ - 7/414720? + 1/103680?V7,4 

+ l/3317760(-2004?2 + 221 + 1216?^ + 566? + 124?V7,i4 
+ 1/3317760(132?^ + 19-86?- 112?^ + 44?>7,ii 
+ (-91/8294400 + 229/829440?^ - 43/207360?^ + 1/165888?^ 
+ 1/23040? - 221/2073600?^)e7.io 



30 KANGHAI TAN AND XIAOPING YANG 

+ l/8294400(-3 + 64t^ + 1440?^ - 672?^ + 630?^ - 1360?^ - 1140e7,2 

+ l/2764800(-143 - 1280?^ + 1840f^ - BeOr^ + 620? + 288f^)e7,3 

+ 1/2764800(109 - 460?^ + 80f^ + 500?^ - 440? + 216?^)e7,i6 

+ l/829440(-49 - 68?^ + 243?^ - 64? - 74/'^)e7j7 

+ (-1 1/414720 + 7/138240?^- 11/207360?^ + 1/460800^7,18 

+ l/3317760(-75 + 96?^ - 1008f^ + 1696?^ - 1332?^ + 510?+ 128?'')e7,i2 

+ (-67/4147200 + 53/276480?^ - l/4608f^ + 31/829440?^ 

+ 1/20736? - 79/2073600?^)e7,i3 
+ (1/41472 + 13/138240?^ - l/41472f^ - 11/1382400^7,15 
+ (1/1935360 + 1/30240?^ - l/10080f^ + 1/8064?^ 

- 1/12096?^ + 1/32256?^ - 1/1612800^7,1 

+ 1/116121600(3741 - 74784?^ + 149040?^ - 164960?^ 
+ 100320?^ - 30906? + 18176?'')e7,5 

+ ^.^^!o^^^ (-12349 - 114640?^ + 188320?^ 
232243200 

- 170200?^ + 75700? + 36544f^)e7,6 

+ 1/11612160(183 - 816?^ + 534?^ - 418? + im^)ejj 
+ (89/1935360 + 223/967680?^ - 41/967680?^ - 247/12902400^7,8 
+ (-1/16128 + 11/80640?- Il/322560?^)e7,9- 
For ? e [0, 1 /2) we obtain 

(6.25) i'^(0 = i'^(l/2),/= I,--- ,7 
and 

P^(0 = (5/2304?4 - 1/1152?^ - 1/921600+ 1/23040?- Il/7200?^)e6,5 

(6.26) + (1/1382400 - 1/17280?^ + 1/5760?^ - 1 /7200?^)e6,4 
-7/138240^6,2 - 1/15360^6,1 - 1/46080^6,8 - 1/92160^6,3 - 1/92160^6,7 

Pl(t) = (-7/25920?^ - 127/232243200 + 1/69120?^ - 5/27648?^ + 1 l/28800?^)e7,9 
+ (1/207360?^ - 1/34560?^ + 1/17280?^ - 1/25920?^ - 1/387072)^7,8 

(6.27) - 1/1105920^7,4 + 1/368640^7,14+ 1/110592067,11 -7/1658880^7,16 
+ 13/3317760^7,13 - 1/552960^7,17 - 1/1935360^7,6 - 1/331776^7,3 
+ 5/442368^7,12 + l/3317760e7,io - 29/15482880^7,5 - 1/552960^7,2 

+ 563/116121600^7,1 - 11/7741440^7,7 - 1/387072^7,18 - 1/322560^7,15 

For s = I,-- - ,7,j= I,-- - , dim V, set A^j =< A*, e^j >=< A^, e^j >■ If (7, A*) satisfies 
the system (14.151) of abnormal equations, then we have V? 6 [0, 1] 

<A*,P]{t) >=0,i=l,2,s=l,--- ,7. 
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Now we can deduce that Asj = for = 1, ■ • ■ , 6 and j = 1, • ■ • , dim V'\ from the fact 
that Pfs are polynomials of the variable t. In fact, from (16.1 1| )- (I6. 141 ) and (|6.18I )- (I6.21| ) it is 
easy to see that = for s = 1 , • ■ • ,4. For s = 5, from (|6.15l) and (16.221) we have Asj = 
and 

f -1/1152^5,4-5/1152^5,2 = 

1/1440^5,5 + 1/192^52 = 
. 1/1920^5,4 - 1/576^5,2 - 1/11520^5,5 + 7/11520^5,3 + 1/11520^5,6 = , 

5/1728^5,4 - 19/1728^5,2 = 

- 1/128^5,4 - 1/128^,5 + 23/1152/15,2 - 5/1152^5,3 = 

from which we then obtain /I5 = 0. 
For s = 6, from (|6.26l) we have 

1/5760^,4 + 5/2304^,5 = 0, 

-1/7200^6,4 - 11/7200/16,5 = 0, 

which implies that /l6,5 = /l6,4 = 0. By (16.161 ) and (16.231 ). we obtain A(,j = and 



(6.28) -7^,2 + 4U6,7 = 0, 

(6.29) -^6,2 + 2^6,8 + 13^6,7 - 10^,6 - ^,3 = 0, 

(6.30) -1 U6,2 - 6^6,8 - 7^,7 + 15^,6 - 3^,3 = 0, 

(6.31) -31/17280/16,7 + 13/8640^6,2 = 0, 

(6.32) l/34560(- 134/16,2 - lA^,^ + 2\9A(,j + 45/^6,6 + 23/^6,3) = 0, 

(6.33) -131/16,2 - 105/^6,8 + 214/16,7 + 45/^6,6 + 36/16,3 - 3/16,9 = 0. 



It follows /J6,2 = A(,j = from (1628]) and (lOTl) . Therefore (l629l) . (lOOl) and (l632l) imply 
^^6,8 = ^,6 = ^,3 = and (16.331) gives Ae,9 = 0. 

Now let us consider s = 1. Recall that dim = 18. However 

<Aj,P]{t) >= 0,/= 1,2, V? 6 [0,1], 

can give at most 16 linear conditions for Aj. In fact, (16.271) gives three independent condi- 
tions: /i7,9 = /l7,8 = and a linear equation for Aj. (16.171) and (|6.24l) give at most 13 linear 
conditions, because (16.171 ). (16.241 ) are 5,6 order polynomials (with respect to t) respectively. 
Note that (16.251 ) does not give more constraints than (16.241 ). □ 

The computation in the proof of Theorem 16.31 is carried out with the most help of Lie 
Tools Package for Maple developed by Miguel Torres-Torriti. The package and a user's 
guide is available at http ://www. cim.mcgill . ca/~migueltt/ltp/ltp.html[ 

From Theorem 16. 3[ Theorem 13.21 and Lemma 13.41 we immediately have the following 
result which generalizes a result by [|27l . 

Corollary 6.4. Any abnormal minimizer in a subriemannian manifold (M, A, ^.„.) ofstep< 6 
has no corner type singularities. 
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